A Lyapunov-Based Methodology for Constrained Optimization with Bandit Feedback

In a wide variety of applications including online advertising, contractual hiring, and wireless scheduling, the controller is constrained by stringent budget constraint on available resources, which are consumed in random amount each action, stochastic feasibility that may impose important operational limitations decision-making. this work, we consider general model to address such problems, where action returns reward, cost, penalty from an unknown joint distribution, decision-maker aims maximize total reward under B cost time-average penalty. We propose novel low-complexity algorithm based Lyapunov optimization methodology, named LyOn, prove for K arms it achieves square root KBlog(B) regret zero constraint-violation when sufficiently large. The low computational sharp performance bounds LyOn suggest Lyapunov-based design methodology can be effective solving bandit problems.